{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2WGeukhYh9fd"
      },
      "source": [
        "# TFDS CLI\n",
        "\n",
        "TFDS CLI is a command-line tool that provides various commands to easily work with TensorFlow Datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vJLAsn1c0Hxu"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "%env TF_CPP_MIN_LOG_LEVEL=1  # Disable logs on TF import"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r-42ZFIIrgbF"
      },
      "source": [
        "Copyright 2020 The TensorFlow Datasets Authors, Licensed under the Apache License, Version 2.0"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "grQeV-PZroqn"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://www.tensorflow.org/datasets/cli\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/datasets/blob/master/docs/cli.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/datasets/blob/master/docs/cli.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uo-yMd3Zrm_K"
      },
      "source": [
        "## Installation\n",
        "\n",
        "The CLI tool is installed with `tensorflow-datasets` (or `tfds-nightly`)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jeV8_FpsiDwH"
      },
      "outputs": [],
      "source": [
        "!pip install -q tfds-nightly\n",
        "!tfds --version"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hdZiDNR1ijRH"
      },
      "source": [
        "For the list of all CLI commands:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CCJPO_Akij0U"
      },
      "outputs": [],
      "source": [
        "!tfds --help"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fJrFRBDKj0sO"
      },
      "source": [
        "## `tfds new`: Implementing a new Dataset\n",
        "\n",
        "This command will help you kickstart writing your new Python dataset by creating\n",
        "a `<dataset_name>/` directory containing default implementation files.\n",
        "\n",
        "Usage:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "c0Bm7yFCk91Q"
      },
      "outputs": [],
      "source": [
        "!tfds new my_dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OZaDtK0elimF"
      },
      "source": [
        "Will create:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CwSPLFRfli8I"
      },
      "outputs": [],
      "source": [
        "ls -1 my_dataset/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rUsoDbi0moyK"
      },
      "source": [
        "See our [writing dataset guide](https://www.tensorflow.org/datasets/add_dataset)\n",
        "for more info.\n",
        "\n",
        "Available options:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pAWCw-fDkwky"
      },
      "outputs": [],
      "source": [
        "!tfds new --help"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x7996uhD1-GP"
      },
      "source": [
        "## `tfds build`: Download and prepare a dataset\n",
        "\n",
        "Use `tfds build <my_dataset>` to generate a new dataset. `<my_dataset>` can be:\n",
        "\n",
        "* A path to `dataset/` folder or `dataset.py` file (empty for current directory):\n",
        "  * `tfds build datasets/my_dataset/`\n",
        "  * `cd datasets/my_dataset/ && tfds build`\n",
        "  * `cd datasets/my_dataset/ && tfds build my_dataset`\n",
        "  * `cd datasets/my_dataset/ && tfds build my_dataset.py`\n",
        "\n",
        "* A registered dataset:\n",
        "\n",
        "  * `tfds build mnist`\n",
        "  * `tfds build my_dataset --imports my_project.datasets`\n",
        "\n",
        "Note: `tfds build` has useful flags to help prototyping and debuging. See the `Debug & tests:` section bellow.\n",
        "\n",
        "Available options:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IGAl6dw62KNO"
      },
      "outputs": [],
      "source": [
        "!tfds build --help"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "cli.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
